Result of a Pilot External Quality Assessment Scheme for Clinical Diagnosis of Inherited Metabolic Disorders in China.

BACKGROUND: We aimed to evaluate the diagnostic capabilities of Chinese laboratories for inherited metabolic disorders (IMDs) using gas chromatography-mass spectrometry (GC-MS) on urine samples. Meanwhile, based on the result of the pilot external quality assessment (EQA) scheme, we hope to establish a standardized and reliable procedure for future EQA practice. METHODS: We recruited laboratories that participated in the EQA of quantitative analysis of urinary organic acids with GC-MS before joining the surveys. In each survey, a set of five real urine samples was distributed to each participant. The participants should analyze the sample by GC-MS and report the "analytical result", "the most likely diagnosis", and "recommendation for further tests" to the NCCL before the deadline. RESULTS: A total of 21 laboratories participated in the scheme. The pass rates were 94.4% in 2020 and 89.5% in 2021. For all eight IMDs tested, the analytical proficiency rates ranged from 84.7% - 100%, and the interpretational performance rate ranged from 88.2% - 97.0%. The performance on hyperphenylalaninemia (HPA), 3-methylcrotonyl-CoA carboxylase deficiency (MCCD), and ethylmalonic encephalopathy (EE) samples were not satisfactory. CONCLUSIONS: In general, the participants of this pilot EQA scheme are equipped with the basic capability for qualitative organic acid analysis and interpretation of the results. Limited by the small size of laboratories and samples involved, this activity could not fully reflect the state of clinical practice of Chinese laboratories. NCCL will improve the EQA scheme and implement more EQA activities in the future.

External quality assurance in the era of standardization.

Metrology in clinical chemistry aims to ensure the equivalence of measurement results from different in-vitro diagnostic measurement devices (IVD MD) for use in healthcare. The metrological traceability of measurement results to higher-order references is the cornerstone to achieving equivalent results. However, other fundamentals are also needed, including the commutability of reference materials and external quality assessment (EQA) materials for monitoring the equivalence of measurement results at the end-user level. This manuscript summarizes the findings and opinions expressed at the Joint Community for Traceability in Laboratory Medicine (JCTLM) workshop held on December 4-5, 2023. The workshop explored the relationship between EQA/proficiency testing and metrological traceability to higher-order references. EQA monitors the equivalence of measurement results from end-user IVD MDs. The workshop discussed the role and challenges of using EQA to improve and maintain the equivalence of measurement results. It also elucidated current developments in establishing the clinical suitability of laboratory results expressed as analytical performance specifications (APS).

Radiotherapy trial quality assurance processes: a systematic review.

Quality assurance remains a neglected component of many trials, particularly for technical interventions, such as surgery and radiotherapy, for which quality of treatment is an important component in defining outcomes. We aimed to evaluate evidence for the processes used in radiotherapy quality assurance of clinical trials. A systematic review was undertaken focusing on use of a pre-trial outlining benchmark case and subsequent on-trial individual case reviews of outlining for recruited patients. These pre-trial and on-trial checks are used to ensure consistency and standardisation of treatment for each patient recruited to the trial by confirming protocol compliance. Non-adherence to the trial protocol has been shown to have a negative effect on trial outcomes. 29 studies published between January, 2000, and December, 2022, were identified that reported on either outlining benchmark case results or outlining individual case review results, or both. The trials identified varied in their use of radiotherapy quality assurance practices and reporting of outcomes was inconsistent. Deviations from trial protocols were frequent, particularly regarding outlining. Studies correlating benchmark case results with on-trial individual case reviews provided mixed results, meaning firm conclusions could not be drawn regarding the influence of the pre-trial benchmark case on subsequent on-trial performance. The optimal radiotherapy quality assurance processes were unclear, and there was little evidence available. Improved reporting of outcomes from radiotherapy quality assurance programmes is needed to develop an evidence base for the optimal approach to radiotherapy quality assurance in trials.

Feasibility of using a transmission ion chamber for QA tests of medical linear accelerators.

PURPOSE: To study the feasibility of using the Integral Quality Monitoring (IQM) system for routine quality assurance (QA) of photon beams. METHODS: The IQM system is a commercially available dose delivery verification tool, which consists of a spatially sensitive large area transmission ion chamber, mounted on the Linac collimator, and a calculation algorithm to predict the signals in response to radiation beams. By comparing the measured and predicted signals the system verifies the accuracy of beam delivery. The ion chamber unit is a battery powered system including a dual-electrometer, temperature and pressure sensors, and inclinometers. The feasibility of using the IQM system for routine QA tests was investigated by measuring constancy values of beam parameters, with specially designed tests fields, and comparing them with those determined by a conventional system. RESULTS: The sensitivity of the beam output constancy measurements by the IQM system was found to agree with those measured by a Farmer type ion chamber placed in water phantoms to within 0.1% for typical daily output variation of ± 0.5% and ± 1%. The beam symmetry was measured with a 4 cm × 4 cm aperture at multiple off-axis distances and was found to have a highly linear relationship with those measured in a water phantom scan for intentionally introduced asymmetry between -3% and +3%. The beam flatness was measured with a two-field ratio method and was found to be linearly correlated with those measured by water phantom scan. The dosimetric equivalent of a picket fence test performed by the IQM system can serve as a constancy check of the multileaf collimator (MLC) bank positioning test. CONCLUSIONS: The IQM system has been investigated for constancy measurements of various beam parameters for photon beams. The results suggest that the system can be used for most of the routine QA tests effectively and efficiently.

A TPS integrated machine learning tool for predicting patient-specific quality assurance outcomes in volumetric-modulated arc therapy.

PURPOSE: Machine learning (ML) models have been demonstrated to be beneficial for optimizing the workload of patient-specific quality assurance (PSQA). Implementing them in clinical routine frequently requires third-party applications beyond the treatment planning system (TPS), slowing down the workflow. To address this issue, a PSQA outcomes predictive model was carefully selected and validated before being fully integrated into the TPS. MATERIALS AND METHODS: Nine ML algorithms were evaluated using cross-validation. The learning database was built by calculating complexity metrics (CM) and binarizing PSQA results into "pass"/"fail" classes for 1767 VMAT arcs. The predictive performance was evaluated using area under the ROC curve (AUROC), sensitivity, and specificity. The ML model was integrated into the TPS via a C# script. Script-guided reoptimization impact on PSQA and dosimetric results was evaluated on ten VMAT plans with "fail"-predicted arcs. Workload reduction potential was also assessed. RESULTS: The selected model exhibited an AUROC of 0.88, with a sensitivity and specificity exceeding 50 % and 90 %, respectively. The script-guided reoptimization of the ten evaluated plans led to an average improvement of 1.4 ± 0.9 percentage points in PSQA results, while preserving the quality of the dose distribution. A yearly savings of about 140 h with the use of the script was estimated. CONCLUSIONS: The proposed script is a valuable complementary tool for PSQA measurement. It was efficiently integrated into the clinical workflow to enhance PSQA outcomes and reduce PSQA workload by decreasing the risk of failing QA and thereby, the need for repeated replanning and measurements.

Detecting outliers beyond tolerance limits derived from statistical process control in patient-specific quality assurance.

BACKGROUND: Tolerance limit is defined on pre-treatment patient specific quality assurance results to identify "out of the norm" dose discrepancy in plan. An out-of-tolerance plan during measurement can often cause treatment delays especially if replanning is required. In this study, we aim to develop an outlier detection model to identify out-of-tolerance plan early during treatment planning phase to mitigate the above-mentioned risks. METHODS: Patient-specific quality assurance results with portal dosimetry for stereotactic body radiotherapy measured between January 2020 and December 2021 were used in this study. Data were divided into thorax and pelvis sites and gamma passing rates were recorded using 2%/2 mm, 2%/1 mm, and 1%/1 mm gamma criteria. Statistical process control method was used to determine six different site and criterion-specific tolerance and action limits. Using only the inliers identified with our determined tolerance limits, we trained three different outlier detection models using the plan complexity metrics extracted from each treatment field-robust covariance, isolation forest, and one class support vector machine. The hyperparameters were optimized using the F1-score calculated from both the inliers and validation outliers' data. RESULTS: 308 pelvis and 200 thorax fields were used in this study. The tolerance (action) limits for 2%/2 mm, 2%/1 mm, and 1%/1 mm gamma criteria in the pelvis site are 99.1% (98.1%), 95.8% (91.1%), and 91.7% (86.1%), respectively. The tolerance (action) limits in the thorax site are 99.0% (98.7%), 97.0% (96.2%), and 91.5% (87.2%). One class support vector machine performs the best among all the algorithms. The best performing model in the thorax (pelvis) site achieves a precision of 0.56 (0.54), recall of 1.0 (1.0), and F1-score of 0.72 (0.70) when using the 2%/2 mm (2%/1 mm) criterion. CONCLUSION: The model will help the planner to identify an out-of-tolerance plan early so that they can refine the plan further during the planning stage without risking late discovery during measurement.

Commissioning of Mobius3D/FX for patient-specific quality assurance: The CUIMC-NewYork Presbyterian Hospital experience.

PURPOSE: To present the process undertaken by our institute in commissioning Mobius3D (M3D) for patient-specific quality assurance. METHOD: 168 plans were randomly selected to compare dose distribution measured with ArcCheck and dose calculated from M3D, both compared against the treatment planning system (TPS). The gamma criteria for measurement and M3D are 3%/2 mm with 10% and 50% dose thresholds, respectively. The effect of tissue inhomogeneity was investigated on 11 plans by recalculating the dose in M3D on a homogeneous phantom. Tolerance and action limits were established following the AAPM Task Group 218 recommendations. RESULTS: The M3D showed high variability in gamma passing rate compared to the measurement. Twenty-three out of 168 plans had false negative dose comparisons. These plans fall under high tissue inhomogeneity like lung and metal implants, small field targets, and breast plans planned with high energy. One false negative case (0.6%) was observed. A single tolerance limit of 91% and 92% gamma passing rate for the M3D and measurement-based PSQA were established, respectively. Against the expectation, recalculating plans on the homogeneous phantom in M3D did not necessarily increase the gamma passing rate. These plans have a duty cycle >4.2, and the small field sizes combined with differences in slice thickness contributed to observed dose differences in the homogeneous phantom comparisons. CONCLUSION: Following the commissioning, M3D is adopted in our institute. Currently, the gamma criteria used for measurement and M3D are 3%/2 mm, 40% dose threshold, with gamma passing rates of 92% and 95%, respectively. A higher passing rate for M3D is adopted until more data is available. The combined effect of plan modulation, the field sizes, the tissue inhomogeneity, the dose algorithm, and the volume averaging effect from differences in slice thickness can contribute to the differences in dose in M3D and TPS.

The performance of a new type accelerator uRT-linac 506c evaluated by a quality assurance automation system.

PURPOSE: The purpose of this study was to evaluate the performance of our quality assurance (QA) automation system and to evaluate the machine performance of a new type linear accelerator uRT-linac 506c within 6 months using this system. METHODS: This QA automation system consists of a hollow cylindrical phantom with 18 steel balls in the phantom surface and an analysis software to process electronic portal imaging device (EPID) measurement image data and report the results. The performance of the QA automation system was evaluated by the tests of repeatability, archivable precision, detectability of introduced errors, and the impact of set-up errors on QA results. The performance of this linac was evaluated by 31 items using this QA system over 6 months. RESULTS: This QA system was able to automatically deliver QA plan, EPID image acquisition, and automatic analysis. All images acquiring and analysis took approximately 4.6 min per energy. The preset error of 0.1 mm in multi-leaf collimator (MLC) leaf were detected as 0.12 ± 0.01 mm for Bank A and 0.10 ± 0.01 mm in Bank B. The 2 mm setup error was detected as -1.95 ± 0.01 mm, -2.02 ± 0.01 mm, 2.01 ± 0.01 mm for X, Y, Z directions, respectively. And data from the tests of repeatability and detectability of introduced errors showed the standard deviation were all within 0.1 mm and 0.1°. and data of the machine performance were all within the tolerance specified by AAPM TG-142. CONCLUSIONS: The QA automation system has high precision and good performance, and it can improve the QA efficiency. The performance of the new accelerator has also performed very well during the testing period.

Is it necessary to perform measurement-based patient-specific quality assurance for online adaptive radiotherapy with Elekta Unity MR-Linac?

This study aimed to investigate the necessity of measurement-based patient-specific quality assurance (PSQA) for online adaptive radiotherapy by analyzing measurement-based PSQA results and calculation-based 3D independent dose verification results with Elekta Unity MR-Linac. There are two workflows for Elekta Unity enabled in the treatment planning system: adapt to position (ATP) and adapt to shape (ATS). ATP plans are those which have relatively slighter shifts from reference plans by adjusting beam shapes or weights, whereas ATS plans are the new plans optimized from the beginning with probable re-contouring targets and organs-at-risk. PSQA gamma passing rates were measured using an MR-compatible ArcCHECK diode array for 78 reference plans and corresponding 208 adaptive plans (129 ATP plans and 79 ATS plans) of Elekta Unity. Subsequently, the relationships between ATP, or ATS plans and reference plans were evaluated separately. The Pearson's r correlation coefficients between ATP or ATS adaptive plans and corresponding reference plans were also characterized using regression analysis. Moreover, the Bland-Altman plot method was used to describe the agreement of PSQA results between ATP or ATS adaptive plans and reference plans. Additionally, Monte Carlo-based independent dose verification software ArcherQA was used to perform secondary dose check for adaptive plans. For ArcCHECK measurements, the average gamma passing rates (ArcCHECK vs. TPS) of PSQA (3%/2 mm criterion) were 99.51% ± 0.88% and 99.43% ± 0.54% for ATP and ATS plans, respectively, which were higher than the corresponding reference plans 99.34% ± 1.04% (p < 0.05) and 99.20% ± 0.71% (p < 0.05), respectively. The Pearson's r correlation coefficients were 0.720 between ATP and reference plans and 0.300 between ATS and reference plans with ArcCHECK, respectively. Furthermore, >95% of data points of differences between both ATP and ATS plans and reference plans were within ±2σ (standard deviation) of the mean difference between adaptive and reference plans with ArcCHECK measurements. With ArcherQA calculation, the average gamma passing rates (ArcherQA vs. TPS) were 98.23% ± 1.64% and 98.15% ± 1.07% for ATP and ATS adaptive plans, separately. It might be unnecessary to perform measurement-based PSQA for both ATP and ATS adaptive plans for Unity if the gamma passing rates of both measurements of corresponding reference plans and independent dose verification of adaptive plans have high gamma passing rates. Periodic machine QA and verification of adaptive plans were recommended to ensure treatment safety.

A Nationwide Network of Health AI Assurance Laboratories.

Importance: Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed. Observations: While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings. Conclusion and Relevance: The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

The Quality Assurance of a 1.5 T MR-Linac.

The recent introduction of a commercial 1.5 T MR-linac system has considerably improved the image quality of the patient acquired in the treatment unit as well as enabling online adaptive radiation therapy (oART) treatment strategies. Quality Assurance (QA) of this new technology requires new methodology that allows for the high field MR in a linac environment. The presence of the magnetic field requires special attention to the phantoms, detectors, and tools to perform QA. Due to the design of the system, the integrated megavoltage imager (MVI) is essential for radiation beam calibrations and QA. Additionally, the alignment between the MR image system and the radiation isocenter must be checked. The MR-linac system has vendor-supplied phantoms for calibration and QA tests. However, users have developed their own routine QA systems to independently check that the machine is performing as required, as to ensure we are able to deliver the intended dose with sufficient certainty. The aim of this work is therefore to review the MR-linac specific QA procedures reported in the literature.